Conjugate Gradient Temporal Difference Learning for Visual Robot Homing

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Deep Feature-Action Processing with Mixture of Updates

© Springer International Publishing Switzerland 2015. This paper explores the possibility of combining an actor and critic in one architecture and uses a mixture of updates to train them. It describes a model for robot navigation that uses architecture similar to an actor-critic reinforcement learning architecture. It sets up the actor as a layer seconded by another layer which deduce the value function. Therefore, the effect is to have similar to a critic outcome combined with the actor in one network. The model hence can be used as the base for a truly deep reinforcement learning architecture that can be explored in the future. More importantly this work explores the results of mixing conjugate gradient update with gradient update for the mentioned architecture. The reward signal is back propagated from the critic to the actor through conjugate gradient eligibility trace for the second layer combined with gradient eligibility trace for the first layer. We show that this mixture of updates seems to work well for this model. The features layer have been deeply trained by applying a simple PCA on the whole set of images histograms acquired during the first running episode. The model is also able to adapt to a reduced features dimension autonomously. Initial experimental result on real robot shows that the agent accomplished good success rate in reaching a goal location

Visual robot homing using Sarsa(λ), whole image measure, and radial basis function

Abstract—This paper describes a model for visual homing. It uses Sarsa(λ) as its learning algorithm, combined with the Jeffery Divergence Measure (JDM) as a way of terminating the task and augmenting the reward signal. The visual features are taken to be the histograms difference of the current view and the stored views of the goal location, taken for all RGB channels. A radial basis function layer acts on those histograms to provide input for the linear function approximator. An onpolicy on-line Sarsa(λ) method was used to train three linear neural networks one for each action to approximate the actionvalue function with the aid of eligibility traces. The resultant networks are trained to perform visual robot homing, where they achieved good results in finding a goal location. This work demonstrates that visual homing based on reinforcement learning and radial basis function has a high potential for learning local navigation tasks. I

Diabetic Retinopathy Detection Using Transfer and Reinforcement Learning with Effective Image Preprocessing and Data Augmentation Techniques

Diabetic retinopathy is the consequence of advanced stages of diabetes, which can ultimately lead to permanent blindness. An early detection of diabetic retinopathy is extremely important to avoid blindness and to recover from it as soon as possible. This chapter discusses the application of recent deep and transfer learning models for medical image analysis, with the focus on diabetic retinopathy detection. The chapter presents an extensive discussion on the publicly available datasets with diabetic retinopathy images, and the Kaggle dataset is used for training and testing of our proposed model. The main challenges to handle noisy and not large enough datasets are discussed in this chapter as well, where image preprocessing techniques and data augmentation play a significant role. An extensive overview of recent data augmentation techniques is also given to tackle the problem of imbalanced nature of diabetic retinopathy datasets. The proposed model integrates deep learning and reinforcement learning to perform detection and imbalanced classification on the Kaggle dataset